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Response to Amendment/Arguments 

1 . This action is in response to amendment filed 08/03/06. 

2. Claims 5, 7, 13, 16, 17, 19, 22-24 and 26 have been cancelled; claims 1-4, 6, 8- 
12,14, 15, 18, 20, 21, 25 and 27-33 are still pending. 

3. Applicant's arguments filed 08/03/06 have been fully considered but they are not 
persuasive; see below. [Note that Applicant's arguments on PP. 13-20 are essentially 
the same as those presented in response filed 03/23/06 (no new arguments are 
presented) and Applicant is directed to the Office action mailed 05/03/06 for Examiner's 
response to these arguments. Below only arguments presented on PP. 9-12 are 
addressed.] 

4. In remarks Applicant argues in substance: 

4.1 that Fowler does not disclose that the lofargram image is enhanced to achieve 
target levels for a mean value or a variation value in human faces (P. 10, 2 nd 
paragraph; P. 12, 2 nd paragraph from bottom) 
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However, Fowler discloses enhancing lofargram images in such a way that the 
each and every strip of the image is enhanced to have a target level of mean 
value [Fig. 1, refs. 3-7; Col. 3, lines 15-30; Col. 6, line 48-Col. 7, line 5. Note that 
here Fowler discloses enhancing all regions of an image such that each region 
has a target level for the mean value; in the case of an image containing faces, 
all facial regions and the non-facial region as a whole are all regions of the image 
(possibly as a result of some segmentation process). Therefore Fowler teaches 
enhancing the entire image.] In addition, Kado discloses automatically 
measuring brightness (i.e., lightness levels) of human faces [Fig. 14, ref. 16 and 
col. 7, lines 33-36] and automatically enhances the image's brightness based on 
the measure brightness [col. 7, lines 44-46]. Since Lofargram images are 
images, Fowler's technique is applicable to enhance images containing human 
faces. 



4.2 that the target level of mean value or variation value are desired lightness and 
contrast levels as determined through a determination of human visual 
preferences (P. 10, last line through P. 11, 1 st paragraph; P. 11, 2 nd paragraph, 
last 2 lines; P. 1 1, 3 rd paragraph, lines 6-8) 



However, the mean or variation values being determined through a determination 
of human visual preferences is not a claim limitation. (In any event, it is admitted 
prior art per page 5, lines 13-16 of the specification and since it is only used to 
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set the target value, Fowler's approach certainly can be applied to enhance the 
image so that each face has such a target value.] Note also that when lightness 
is adjusted, contrast may change accordingly. In any event, per the analysis in 
the rejection of claims 30 and 31, Fowler also discloses adjusting the pixel values 
to achieve a desired variance value, which results in a different contrast. 

4.3 that it seems highly unlikely for one of ordinary skill in the art of facial recognition 
and enhancement to look at the art of sonar images and the combination was 
based on improper hindsight reasoning (P. 12, 3 rd paragraph) 

In response to applicant's argument that the examiner's conclusion of 
obviousness is based upon improper hindsight reasoning, it must be recognized 
that any judgment on obviousness is in a sense necessarily a reconstruction 
based upon hindsight reasoning. But so long as it takes into account only 
knowledge which was within the level of ordinary skill at the time the claimed 
invention was made, and does not include knowledge gleaned only from the 
applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 
443 F.2d 1392, 170 USPQ 209 (CCPA 1971). In this case, the three references 
are all from the same field of endeavor of image enhancement and the specific 
enhancing techniques disclosed in Kato and Fowler can be added to Schildkraut 
to obtain the invention as specified in the claims, namely, using Kato's teaching 
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to automatically enhance the brightness and further using Fowler's teaching to 
obtain a target mean brightness level in the facial region. 



Claim Rejections - 35 USC § 103 



5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a 
person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived 
by the manner in which the invention was made. 



6. Claims 1, 2, 8, 10, 15, 21 and 27-33 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Schildkraut et al. (US 6,292,574), in view of Kado et al. (US 
6,181,806) and Fowler (US 5,410,618). 



7. Regarding claim 1, and similarly claims 8, 15 and 21 (all as amended), 
Schildkraut discloses: 

• automatically detecting one or more human faces in an image using face detection algorithms 
and automatically locating the one or more human faces in the image 
[Fig. 2, numeral S10; Fig. 6; Col. 4, line 13-Col. 5, Iine4] 



Schildkraut does not expressly disclose 

• automatically enhancing an appearance of the entire image by using a mapping technique to 
produce the image with target levels for a mean value or a variation value of the pixels in the one 
or more human faces, wherein the entire image is automatically enhanced such that the pixels in 
the one or more human faces have the target levels for the mean value or the variation value of 
the pixels 
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However, Kado discloses automatically measuring brightness (i.e., lightness levels) of 
human faces [Fig. 14, ref. 16 and col. 7, lines 33-36] and automatically enhances the 
image's brightness based on the measure brightness [col. 7, lines 44-46]. In addition, 
Fowler discloses linear mean invariant transforms (i.e., a mappings) that enhance 
lofargram images in such a way that the each and every strip of the image is enhanced 
to have a target level of mean value [Fig. 1, refs. 3-7; Col. 1, lines 30-40; Col. 3, lines 
15-30; Col. 6, line 48-Col. 7, line 5. Note that here Fowler discloses enhancing all 
regions of an image such that each region has a target level for the mean value; in the 
case of an image containing faces, all facial regions and the non-facial region as a 
whole are all regions of the image (possibly as a result of some segmentation process). 
Therefore Fowler teaches enhancing the entire image.] [For demonstration purpose 
(but not relied upon for rejection) an example of a lofargram image can be found in the 
first slide of the presentation made by Vincent Premus in the 7 th Annual ASAP '99 
Workshop.] 

Schildkraut, Kado and Fowler are combinable because they ail have aspects that are 
from the same field of endeavor of image enhancement (note that red eye correction is 
a form of image enhancement). 

At the time of the invention, it would have been obvious to one of ordinary skill in the art 
to modify Schildkraut with the teachings of Kado and Fowler to enhance detected 
human faces by using a mapping technique. The motivation would have been to 
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improve visual quality, among other things, as Fowler indicated in [Col. 1, lines 11-20 
and Col. 2, lines 28-35]. In addition, as the purpose of red eye correction is to provide 
an facial image with a better visual quality, it would have been obvious to go one step 
further by enhancing the overall visual quality of the faces (and not just the eyes) in the 
image since doing so would provide even better results to the clients (e.g., by a photo- 
processing lab to its customers). 

Therefore, it would have been obvious to combine Kado and Fowler with Schildkraut to 
obtain the invention as specified in claim 1 . 

8. Regarding claim 2, and similarly claim 10, Kado further discloses 

• wherein the module for enhancing the appearances of the image includes a module for 
automatically enhancing lightness levels of the images to enhance the appearance of the one or 
more human faces 

[Fig. 14, ref. 16 and col. 7, lines 44-46] 

9. Regarding claim 27, and similarly claims 28 and 29, Schildkraut further discloses 

• automatically locating eyes in the human faces 
[Fig. 2, numerals S16-S30; Figs. 9, 11] 

10. Regarding claims 30 and 31, Fowler further discloses mapping using the recited 
equations. [Col. 6, line 48-Col. 7, line 5. Note that Eq. 13 is a more general mapping 
function that includes both scaling and offsetting. When only offsetting is performed, 
i.e., A = 1, Eq. 13 becomes P + B (note that in line 1 of col. 7 "w" should have been V). 
On the other hand, when only scaling is performed, i.e., B = 0, Eq. 13 becomes 
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11. AP + (1-A)u. Note further that B is the difference of the new and the original 
means and A is the square root of the ration of the new and the original variants, per 
Eqs. 11 and 12 in col. 6] 

12. Regarding claims 32 and 33, note that as admitted in page 5, lines 13-18 of the 
specification, determining the target levels through a determination of human visual 
preferences is well known in the art and would obviously have been considered by one 
of ordinarily skill in the art since facial images, especially photographs, are typically 
viewed by humans. 

*** 

13. Claims 3, 4, 11, 12 and 18 are rejected under 35 U.S. C. 103(a) as being 
unpatentable over Schildkraut et al. (US 6,292,574), Kado et al. (US 6,181,806) and 
Fowler (US 5,410,618) as applied to claims 1, 2, 8, 10, 15, 21 and 27-33 above, further 
in view of Center, Jr. et al. (US 6,680,745). 

Regarding claims 3 and 4, and similarly claims 11, 12 and 18, the combined invention of 
Schildkraut, Kado and Fowler discloses everything except the following, which Center 
teaches: 

• wherein the module for enhancing the appearances of the image includes a module for 
automatically enhancing contrast levels (claim 3) or color levels (claim 4) of the images to 
enhance the appearance of the one or more human faces 

[Fig. 1; col. 2, lines 55-60. Note that while Center discloses adjusting the camera's settings, one 
of ordinary skill in the art would have recognized that such changes can be achieved by image 
processing means; see, for example, the analysis of claim 2 above in which Kado is relied upon 
for enhancing brightness] 
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The combined invention of Schildkraut, Kado and Fowler is combinable with Center 
because they have aspects that are from the same field of endeavor of face detection. 

Therefore, at the time of the invention it would have been obvious to one of ordinary skill 
in the art to modify the combined invention of Schildkraut, Kado and Fowler with the 
teaching of Center by enhancing an appearance of the image by changing the contrast 
and/or color (in addition to changing brightness as Kado disclosed). The motivation 
would have been to obtain better images, as Center indicated in column 2, lines 55-60. 

Therefore, it would have been obvious to combine Center with Schildkraut, Kado and 
Fowler to obtain the inventions as specified in claims 3 and 4, respectively. 

*** 

14. Claims 6, 14, 20 and 25 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Schildkraut et al. (US 6,292,574), Kado et al. (US 6,181,806) and 
Fowler (US 5,410,618) as applied to claims 1, 2, 8, 10, 15, 21 and 27-33 above, further 
in view of Acker et al. (US 6,009,209). 

Regarding claim 6, and similarly claims 14, 20 and 25, the combined invention of 
Schildkraut, Kado and Fowler discloses all limitations except the following, which Acker 
et al. teaches: 

• reducing or removing the red eye artifact from the human faces 
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[Fig. 5, numeral 109; Fig. 9, numeral 504; Fig. 11; Fig. 13] 

The combined invention of Schildkraut, Kado and Fowler is combinable with Acker 
because they have aspects that are from the same field of endeavor of red eye 
detection. 

Therefore, at the time of the invention it would have been obvious to one of ordinary skill 
in the art to modify the combined invention of Schildkraut, Kado and Fowler with the 
teaching of Acker et al. by detecting and removing red-eye effects from the image. The 
motivation would have been to remove the unpleasant appearance of red-eye defects in 
an image of a person's face caused by, e.g., a flash when the image was taken in order 
to produce a more natural-looking face. 

Therefore, it would have been obvious to combine Acker et al. with Schildkraut, Kado 
and Fowler to obtain the invention as specified in claim 6. 

Conclusion and Contact Information 

15. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 
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A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

16. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Yubin Hung whose telephone number is (571) 272- 
7451 . The examiner can normally be reached on 7:30 - 4:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jingge Wu can be reached on (571) 272-7429. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Yubin Hung 
Patent Examiner 
Art Unit 2624 
September 15, 2006 




